Building, Evaluating & Interpreting Models Overview

Building, Evaluating and Interpreting Models for Bias and Uncertainty Overview

This lesson contains some really fantastic tools for working with EHR data in AI.

Note: At the time this material was created some of the TensorFlow features are in beta form and may change as they launch. That being said these are some bleeding-edge resources to help with many AI projects.

• In the first part, you will get hands-on with using TensorflowDenseFeatures for building a simple regression model.
• Next, you will first review some common evaluation metrics for EHR models and then learn to implement brier scores for model evaluation
• Then, you'll conduct a demographic bias analysis and become familiar with a framework out of the University of Chicago called Aequitas. You will use this Aequitas for group bias and fairness disparity analysis.
• Next, you will implement uncertainty estimation using the Tensorflow Probability API. You will also review some of the underlying concepts for Bayesian probability and types of uncertainty that very important in evaluating model performance.
  • Note: TensorFlow Probability is still in beta at the time this course was published. Documentation is still being built out and some of the code may change.
• Finally, you will interpret models with Shapley values. You will first review model interpretability and some model agnostic methods. Then you will use one of those methods, Shapley values.
  • Note: This section is included as a bonus and is not included in the project itself, but is highly useful.

There is so much to cover in this section. Head to the next section to get started.